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completely new closed-form exact estimator. 
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1. Introduction 



In the classical statistical estimation problem, the starting point is a family P of probability 
measures depending on the parameter 6 belonging to some subset G of a finite-dimensional 
Euclidean space. Each is the distribution of a random element. It is assumed that a 
realization of one random element corresponding to one value 9 = 6q ol the parameter is 
observed, and the objective is to estimate the values of this parameter from the observations. 
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The intuition is to select the value corresponding to the random element that is most 
likely to produce the observations. A rigorous mathematical implementation of this idea 
leads to the notion of the regular statistical model [lOj: the statistical model (or estimation 
problem) P^, G 0, is called regular, if the following two conditions are satisfied: 

— there exists a probability measure Q such that all measures are absolutely contin- 
uous with respect to Q; 

— the density dP^/dQ, called the likelihood ratio, has a special property, called local 
asymptotic normality. 

If at least one of the above conditions is violated, the problem is called singular. 

In regular models, the estimator 6 of the unknown parameter is constructed by maxi- 
mizing the likelihood ratio and is called the maximum likelihood estimator (MLE). Since, 
as a rule, 6 ^ 6q, the consistency of the estimator is studied, that is, the convergence of 
6 to Oq as more and more information becomes available. In all known regular statistical 
problems, the amount of information can be increased in one of two ways: (a) Increasing 
the sample size, for example, the observation time interval (large sample asymptotic); (b) 
reducing the amplitude of noise (small noise asymptotic). The asymptotic behavior of 9 in 
both cases is well-studied. It is also known that many other estimators in regular models 
are asymptotically equivalent to the MLE. 

While all regular models are in a sense the same, each singular model is different. Some- 
times, it is possible to approximate a singular model with a sequence of regular models. 
For each regular model, an MLE is constructed, and then in the limit one can often get 
the true value of the parameter while both the sample size and the noise amplitude are 
fixed. Some singular models cannot be approximated by a sequence of regular models and 
admit estimators that have nothing to do with the MLE [T3]. In this paper, Section HI we 
introduce a completely new type of such estimators for a large class of singular models. 

Infinite-dimensional stochastic evolution equations, that is, stochastic evolution equa- 
tions in infinite-dimensional spaces, are a rich source of statistical problems, both regular 
and singular. A typical example is the Ito equation 




< 




u{0) = uo 
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where t G [0,T], -^ii ■M.j are linear operators, /, gj are adapted processes, Wj are 
independent Wiener processes, and is the unknown parameter belonging to an open subset 
of the real line. The underlying assumption is that the solution u exists, is unique, and can 
be observed as an infinite-dimensional object for all t £ [0, T]. Depending on the operators 
in the equation, the estimation model can be regular, a singular limit of regular problems, 
or completely singular. 

If Ao,Ai, Mj are partial differential or pseudo-differential operators, (jl.ip becomes 
a stochastic partial differential equation (SPDE), which is becoming increasingly popular 
for modelling various phenomena in fluid mechanics [25j, oceanography [21], temperature 
anomalies [H [22], finance [21 [5l [6], and other domains. Various estimation problems for 
different types of SPDEs have been investigated by many authors: [H HI [71 [51 [9l [Til [El [13 
[IZl[IHl[ia etc.]. 

Depending on the stochastic part, (jl.ip is classified as follows: 

— equation with additive noise, if Aij = for all j; 

— equation with multiplicative noise (or bilinear equation) otherwise. 
Depending on the operators, (jl.ip is classified as follows: 

— Diagonalizable equation, if the operators -^i; and Mj, J > 1, have a common 
system of eigenfunctions h^, k > 1, and this system is an orthonormal basis in a 
suitable Hilbert space. 

— Non-diagonalizable equation otherwise. 

A diagonalizable equation is reduced to an infinite system of uncoupled one-dimensional 
diffusion processes; these processes are the Fourier coefficients of the solution in the basis 
/ifc. As a result, while somewhat restrictive as a modelling tool, diagonalizable equations 
are an extremely convenient object to study estimation problems and often provide the 
benchmark results that carry over to more general equations. 

The parameter estimation problem for a diagonalizable equation (jl.ip with additive 
space-time white noise (that is, gj = hj and Aij = for all j) was studied for the first time 
by Huebner, Khasminskii, and Rozovskii [8j, and further investigated in [71 |8l |9l [23] . The 
main feature of this problem is that every A^-dimensional projection of the equation leads to 
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a regular statistical problem, but the problem can become singular in the limit ^ oo (a 
singular limit of regular problems); when this happens, the dimension of the projection 
becomes a natural asymptotic parameter of the problem. Once the diagonalizable model is 
well- understood, extensions to more general equations can be considered ([ISIIE]). 

This paper is the first attempt to investigate the estimation problem for infinite-dimen- 
sional bilinear equations. Such models are often completely singular, that is, cannot be 
represented as a limit of regular models. We consider the more tractable situation of di- 
agonalizable equations. In Section [2] we provide the necessary background on stochastic 
evolution equations, with emphasis on diagonalizable bilinear equations. The maximum 
likelihood estimator (MLE) and its modifications for diagonalizable bilinear equations are 
studied in Section [3l We give sufficient conditions on operators Ao,Al,^A that ensure 
consistency and asymptotic normality of the MLE. We also demonstrate that the MLE 
in this model is not always the best estimator, which, for a singular model is not at 
all surprising. Section S] emphasizes the point even more by introducing a closed-form 
exact estimator. Due to the specific structure of stochastic term, for a large class of 
infinite-dimensional systems with finite-dimensional noise, one can get the exact value of the 
unknown parameter after a finite number of arithmetic manipulations with the observations. 
The very existence of such estimators in these models is rather remarkable and has no 
analogue in classical statistics. 

As an illustration, let ^ be a positive number, W a standard Wiener process, and consider 
the Ito equation 



du{t, x) — 9uxx{t, x)dt = n(t, x)dW{t), t > 0, x e [0, vr). 




with zero boundary conditions. If /ifc(x) = \plpK sin(/!;x). A; > 1, and 




then 





and each is a geometric Brownian motion: 
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We assume that tifc(O) / for all k > 1. In Sections [3] and H] we establish the following 
result. 

Theorem 1.1. // the solution of equation (jl.2p is observed in the form ()1.3p . then the 
parameter 6 can be computed in each of the following ways: 

(El) e = ^lim In ^ - ^) for every k > 1; 

(E2) e = hm -i- In for every T > 0; 

Both (El) and (E2) are essentially the same maximum likelihood estimator, but the 
infinite-dimensional nature of the equation makes it possible to study this estimator in two 
different asymptotic regimes. (E3) is a closed-form exact estimator. While it is most likely 
to be the best choice for this particular problem, we show in Section U] that computational 
complexity of closed-form exact estimators can dramatically increase with the number of 
Wiener processes driving the equation, while the complexity of the MLE is almost unaffected 
by this number. The result is another unexpected feature of closed-form exact estimators: 
ever though they produce the exact value of the parameter, they are not always the best 
choice computationally. 



2. Stochastic Parabolic Equations 

In this section we introduce the diagonalizable stochastic parabolic equation depending on 
a parameter and study the main properties of the solution. 

Let H be a separable Hilbert space with the inner product (•, •)o and the corresponding 
norm || • ||o. Let A be a densely-defined linear operator on H with the following property: 
there exists a positive number c such that ||Am||o > c||ti||o for every u from the domain 
of A. Then the operator powers A"^, 7 G M, are well defined and generate the spaces H"^: 
for 7 > 0, H''' is the domain of A'''; H*^ = H; for 7 < 0, H'^ is the completion of H with 
respect to the norm || • ||^ := ||A - ||o (see for instance Krein at al. [E]). By construction, the 
collection of spaces {H'^, 7 G M} has the following properties: 
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- A''{W) = W-^ for every 7, r G M; 



— For 71 < 72 the space H'''^ densely and continuously embedded into H'^^ : H'''^ C H"''^ 
and there exists a positive number C12 such that ||n||-y^ < ci2||u||72 for all u G H'^^ ; 

— for every 7 G M and m > 0, the space H'''"™' is the dual of H'^^™ relative to the inner 
product in H'*', with duality (•, •)-y,m given by 

{ui,U2)y,n, = (AT-'"^zi,AT+'"n2)o, where m G H^"™, U2 G H^+'". 

Let {Q,J-,{J-'t},^) be a stochastic basis with the usual assumptions, and let {Wj, j > 
1} be a collection of independent standard Brownian motions on this basis. Consider the 
following Ito equation 

du(i) + (A + QA{)u(t)dt = f{t)dt + {Mju{t) + gk{t))dWj{t), 0<t<T, 

i>i (2.1) 

u{0) = Uq 

where -^i, -Mj are linear operators, / and gk are adapted process, and is a scalar 
parameter belonging to an open set C M. 

Definition 2.1. 

(a) Equation (j2.ip is called an equation with additive noise if Mj = for all j > 1. 
Otherwise, (j2.ip is called an equation with multiplicative noise (also known as a 
bilinear equation^. 

(h) Equation (j2.ip is called diagonalizable if the operators Aq, Ai,Mj^ j > 1, have a 
common system of eigenf unctions {h^, k > 1} such that {/i^, k > 1} is an orthonormal 
basis in H and each hj. belongs to every H''. 

(c) Equation is ca/Zed parabolic in the triple H^, H"^-™) if 

o the operator Aq + 9Ai is uniformly bounded from H'''"'"™ to H'''"™ for 9 £ Q : there 
exists a positive real number Ci such that 

II {Ao + eAl)v\\^^rn < Ci \\v\\^+m (2.2) 



for alieeQ,ve HT+™; 
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o There exists a positive number 6 and a real number C such that, for every v E H'>'+™-^ 

- 2{{Ao + eAi)v, v)^,^ + J2 ll-^j^ll' + '^ll^ll'+m < C\\vf^. (2.3) 

i>i 

Remark 2.2. (a) Note that (j2.2p and (j2.3p imply uniform continuity of the family of 
operators Mj, i > 1 from H'*'"'"™' to H'^; in fact, 

Y^\\M,vf^<2C,\\v\\'^^^ + C\\vf^. 
(b) If equation (|2.ip is parabolic, then condition ()2.3p implies that 

((2A + 20^1 + CI)v, v)^,m > 6\\vfy+m, 

where / is the identity operator. The Cauchy-Schwartz inequality and the continuous 
embedding of 'H."/+"^ into H"^ then imply 

11(2^0 + 2eAi + CI)v\\^ > di\\v\\^ 

for some 6i > uniformly in G 0. As a result, we can take A = {2Ao + 26* Ai + C7/)^/(2"^) 
for some fixed 9* £ Q. 

From now on, if equation ()2.ip is parabolic and diagonalizable, we will assume that the 
operator A has the same eigenf unctions as the operators Aq, Ai, Mj; by Remark \ 2. 21 this 
leads to no loss of generality. 

Example 1. (a) For < i < T and x G (0, 1), consider the equation 

du{t,x) -eu^^{t,x)dt = uS,x)dW{t) (2.4) 

with periodic boundary conditions; Ux = du/dx. Then H'>' is the Sobolev space on the 
unit circle (see, for example, Shubin [26, Section 1.7]) and A = \/I — A., where A is the 
Laplace operator on (0, 1) with periodic boundary conditions. Direct computations show 
that equation (j2.4p is diagonalizable; it is parabolic if and only if 20 > 1. 

(b) Let G be a smooth bounded domain in M'^. Let A be the Laplace operator on G with 
zero boundary conditions. It is known (for example, from Shubin [26]), that 

1. the eigenfunctions {hk, A; > 1} of A are smooth in G and form an orthonormal basis 
in L2(G); 
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2. the corresponding eigenvalues Ufc, k > 1, can be arranged so that < —ai < — cj2 < • • ., 



and there exists a number c > such that |(Tfc| ~ ck'^/'^, that 



IS, 



hm \ak\k ^/"^ = c. 

We take H = L2{G), A = \/I — A., where / is the identity operator. Then ||Ati||o > 
VT^-oTll^illo and the operator A generates the Hilbert spaces H''', and, for every 7 G M, 
the space H''' is the closure of the set of smooth compactly supported function on G with 
respect to the norm 

V^' f 

^{1 + k'^y\ifk\'^ \ , where ifk= fix)hk{x)dx, 

\k>l J •^'^ 

which is an equivalent norm in H'^. Let 9 and a be real numbers. Then the stochastic 
equation 

du - 9 Audi = An dW (2.5) 

is 

— always diagonalizable; 

— parabolic in (H'''+-'^, H''', H'^~^) for every 7 G M if and only if 29 > 1. 
Indeed, we have ^0 = 0, ^1 = —A, Mi = A, Mj = 0, j > 2, and 

-29{Aiv,v)y,i = -29\\vf^^^ + 29\\uf^, 
and so IjOM holds with 5 = 26* - 1 and C = 26^. 



Remark 2.3. Taking in (j2.ip H = L2{G), where G is a smooth bounded domain in M'^, and 
^0 = — -^1 = I, MjU = hk{x)u{x), gk = hk{x)g{t,x), we get a bilinear equation driven 
by space-time white noise. Direct analysis shows that this equation is not diagonalizable. 
Moreover, the equation is parabolic if and only if d = 1, that is, when G is an interval; for 
details, see the lecture notes by Walsh 



For a diagonalizable equation, the parabolicity condition ()2.3p can be expressed in terms 
of the eigenvalues of the operators in the equation. 



Estimation in bilinear equations 

Theorem 2.4. Assume that equation (|2.ip is diagonalizable, and 
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■Aohk — Pkhk, Aihk — i^khk, -Mjhk — fJ-jkhk- 
With no loss of generality (see Remark 12. 2p , we also assume that 

Ahk = Xkhk- 

Then equation (j2.ip is parabolic in the triple (H'^"'''", H"^, H'''"'") if and only if there exist 
positive real numbers 6, Ci and a real number C2 such that, for all k > 1 and 6 £ Q, 

Xf"'\pk + Ouk\<Ci- (2.6) 

- 2ipk + e,^k) + E l/^ifcl' + -^^i" ^ ^2. (2.7) 
i>i 

Proof. We show that, for a diagonahzable equation, (j2.6p is equivalent to ()2.2p and (|2.7p is 
equivalent to (|2.3p . Indeed, note that for every 7, r S M, 

||^A;|l7+r = 1 1 A*" /ifc 1 1 = A^||/lfc||^. 

Then ([22]) is ([22]) with v = hk, and ([IZD is ([231) with v = hk- Since both ([Ml) and I^Jij are 
uniform in k and the collection {/i^. A; > 1} is dense in every H''', the proof of the theorem 
is complete. □ 

The following is the basic existence/uniqueness/regularity result for parabolic equations; 
for the proof, see Rozovskii [24:\ Theorem 3.2.1]. 

Theorem 2.5. Assume that equation p.ip is parabolic in the triple (H'^+"*, H''', H'''"™') 
and 

1. the initial condition uq is deterministic and belongs to H'^ ; 

2. the process f = f{t) is Tt-adapted with values in H'^"'" and 

E \\f{t)f,-radt<<^; 

3. each process Qk = gk{t) is J-t-adapted with values in H'*' and 
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Then there exists a unique Tt-adapted process u = u{t) with the following properties: 

— n G L2(0; L2((0, T); f] L2{n; C((0, T); U^)); 

— u is a solution of (|2.ip . that is, the equality 

u{t) + [ {Ao + 9Ai)u{s)ds = uo+ [ f{s)ds 
Jo Jo 

+ J2{MMs)+gk{s))dWj{s). 

holds in H'^"™' for all t G [0,T] on the same set fi' C of probability one; 

— There exists a positive real number Cq depending only on T and the numbers C, 6 in 
([2:3]) such that 

Ht)\\i+^r 

0<t<T 



E sup ||n(t)||2+E/ \\u{t)f^^„,dt<Co{\\uof^+E f \\f{t)f^_„,dt 

0<t<T 'Jo \ ' Jo 

i>i 



Corollary 2.6. Assume that equation (j2.ip is parabolic and diagonalizable. Then, under 

the assumptions of Theorem 12.51 we have 

00 00 

u{t) = Y,Ukit)hk andj] A^^E|ufc(t)|2 < 00, t G [0,r], (2.8) 

k=l k=l 

where Uk{t) = (A'''u(t), /ifc)o satisfies 

n 

duk{t) = ({pk + Ovk)uk{t) + fk{t))dt + J2iiijkUkit) + gkit))dWjit), (2.9) 

i=i 

with uk{0) = (A^uo,/ifc)o, fk{t) = (AV(t),/ifc)o,m, 9k{t) = {A^g{t),hk)o. 



3. Maximum Likelihood Estimators 

With {Q,J^,{Tt}t>o,^), Wj, j > 1, and {H*", r G M} as in the previous section, consider 
the stochastic Ito equation 

du{t) + {Ao + eAi)u{t)dt = ^Mju{t)dWj{t), 0<t<T, u{0) = uq. (3.1) 
We assume that 
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— equation ()3.ip is parabolic in the triple (H'*'"'"™; H'*', H'*'"™') for some 7 E M, m > 0; 



— equation ()3.ip is diagonalizable; 

— The solution of (j3.ip is observed (can be measured without errors) for all t G [0, T]. 

The objective is to estimate the real number 9 from the observations u{t), t G [0, T]. 

Even though whole random field u can be observed, the actual computations can be 
performed only on a finite-dimensional projection of u. By Corollarv 12.61 we have 



u{t) = Y,Uk{t)hk, (3.2) 

k=l 

Uk{t)+[ {pk + Ouk)uk{s)ds = {A'^uo,hk)o + [ Uk{s)y2 fJ^jkdWjis), (3.3) 
Jo Jo ^ 

Thus, a finite collection of the Geometric Brownian motions Uk is a natural finite-dimen- 
sional projection of u. 

To simplify certain formulas, we will use the following notations: 

Mk = J2\tijk\^ Vk = ^. (3.4) 

i>i 



3.1. Maximum Likelihood Estimator (MLE) 

Let Uk^, ■ ■ ■ ,Ukj^ be a finite collection of diffusion processes (j3.3p . For each 9 £ Q, the 
vector Un = {uk^ , • • • , Wfcjv ) generates a measure on the space of continuous M^-valued 
functions. If these measures are absolutely continuous with respect to some convenient 
reference measure, then the MLE of 9 will be the value maximizing the corresponding 
density given the observations. The choice of the reference measure is dictated, among 
other factors, by the possibility to find a closed-form expression of the density. For diffusion 
processes with a parameter in the drift, the standard choice is the measure generated by 
the process with a fixed value of the parameter, for example, the true value ^o- Analysis of 
the relevant conditions for mutual absolute continuity, as given, for example, in the book 
by Liptser and Shiryaev [HI Theorem 7.16], demonstrates that 
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— if A^=l, then the measures generated by for different values of 9 are mutually 
absolutely continuous, and the density with respect to the measure corresponding to 
the true parameter 9q is 

LkiO, Oo) = exp I - / -^^^ - ^ - P^^^^' - ^ - - ^ }■ (3.5) 







— For > 1, the measures are typically mutually singular and so is the resulting estima- 
tion problem. We will see later how to exploit this singularity and gain a computational 
advantage over the straightforward MLE. 

Thus, observation of a single process Uk{t), < t < T, provides an MLE 9k of 9; by 

33, 





By Ito's Lemma, 



and hence from ()3.6p we get 



dln(ufc) = — - IrMkdt, 
Uk 2 



1 Uk(O) Mk , , 

In-^ -. (3.7) 



i/fcT Ufc(T) Ivk 

Notice that, by uniqueness of solution of equation (j3.3p . the function Ukii) cannot change 
sign and so Uk{0)/uk{T) > 0. From (j3.6p and (j3.3p we have the following alternative 
representation of the MLE: 

^^ „ 1 



in particular, 



Y.HkW,{T); (3.8) 



E(4 - ^o)' = Y (3-9) 



and y/T/rjk {9k — 9q) is a standard Gaussian random variable for every T > and k > 1. 

All properties of the MLE (j3.7p now follow directly from (|3.8p and (|3.9p and are sum- 
marized below. 



Theorem 3.1. Assume that equation ()3.ip zs diagonalizable, parabolic in the triple 
(H^+™; H^, HT-'") /or some 7 G M, m > 0, and G H^. T/ien 
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1. For every k>l and T > 0, 6k is an unbiased estimator of 9q. 



2. For every k > 1, as T ^ oo, 9^ converges to 9q with probability one and \'T{9k — 9q) 
converges in distribution to a Gaussian random variable with zero mean and variance 



3. If, in addition, 



lim?7fc = 0, (3.10) 

fe>i 



then, for every T > 0, as k ^ oo, 9^ converges to 9q with probability one and {9k — 
^o)/\/% converges in distribution to a Gaussian random variable with zero mean and 
variance 1/T. 

Remark 3.2. Conditions (12. 7p and (j3.10p are, in general, not connected. Indeed, let A = 
\// — A, where A is the Laplace operator on a smooth bounded domain in M*^ with zero 
boundary conditions. Then equation 

du - (Au - 9u)udt = KudW{t) 

satisfies (j2.3p . but does not satisfy (j3.10p : in this case, Wva.k^oo'Hk = oo. Similarly, equation 

du - {9Au - u)dt = (/ - Af/^udW{t) 

does not satisfy (j2.3p for any 9, but satisfies (j3.10p . We remark that the solution of this last 
equation can be constructed in special weighted Wiener chaos spaces that are much larger 
than L2(l^;L2((0,r);H'>')); see [20]. 



Example 2. Let us consider the following modification of equation ()2.5p from Example 

du - {Au + 9u)dt = ^{l- A)-^''^udWj{t) . 
i>i 

We have Vk = ^, Pk = —o'k > 0, where ak are the eigenvalues of A, and so pk ~ 
Pjk = (1+ Pk)'^ and 

Mk = Yl n / M = - ^ 0, ^ oo. 
(1 + PkV Pk 

By Theorem 13 . 1 1 the maximum likelihood estimator 9k of 9 is 

1 1 MO) ^ 1 

T Uk{T) 2ak 
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IE(4 - Oof ~ cT-^k-^/'^ 



3.2. Modifications of the MLE 

By Theorem 13. H the MLE (|3.7p can be consistent and asymptotically normal either in the 
limit T ^ oo or in the limit k ^ oo. An increase of T always improves the quality of the 
estimator by reducing the variance; if (j3.10p holds, then the variance of the estimator can 
be further reduced by using Uk with the largest available value k. 

The natural question is whether the quality of the estimator can be improved even more 
by using more than one process Uk- This question is no longer of statistical nature: as 
equation (|3.3p shows, each Uk contains essentially the same stochastic information. More 
precisely, the sigma-algebra generated by each Uk{t), t S [0, T] coincides with the sigma- 
algebra generated by ^jkWj{t), j >1, t £ [0, T] (some of fijk can, in principle, be zeroes). 
Moreover, as was mentioned above, the statistical estimation model for 6, involving two or 
more processes Uk, is singular. In what follows, we will see how to use this singularity to 
gain computational advantage over (j3.7p . 

The problem can now be stated as follows: given a sequence of numbers 9k such that 
lim/j^oo ^fc = ^0) can we transform it into a sequence 6^ such that 

lim 9k = 9q, limsup^; < 1. (3.11) 

If (j3.1ip holds, it is natural to say that 9k converges to 9q faster than 9k- Accelerating the 
convergence of a sequence is a classical problem in numerical analysis. The main features of 
this problem are (a) There are many different methods to accelerate the convergence, and 
(b) the effectiveness of every method varies from sequence to sequence. 
We will investigate two methods: 

1. Weighted averaging; 



2. Aitken's method. 
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Theorem 3.3 (Weighted averaging). Let (3k, k > 1, be a sequence of non-negative numbers 
and 

^f3k = +00. 

k>l 

Define the weighted averaging estimator ^^^-j by 

N 

9iN) = '-^ . (3.12) 

k=l 

Then 

1. For every N >1 and T > 0, ^(^) is an unbiased estimator of 9q. 

2. For every N > 1, asT ^ oo, 0{n) converges to Oq with probability one and -\/T'(^(7V) ~^o) 
converges in distribution to a Gaussian random variable with zero mean and variance 

^ ^ /E|Li(AmM y. (3.13) 

j>l V J2k=l Pk J 

3. If, in addition, (|3.10p holds then, for every T > 0, as N ^ oo, ^(jy) converges to 9q 
with probability one. 



Proof. By (pTIS]) . 

E [T.{Pkhijkli^k)]wj{T) 

9iN)=0o + '-^^^ ^ ^ , (3.14) 

TEPk 

k=l 

from which the first two statement of the theorem follow. For the last statement, we combine 
(|3.12|) with the Toeplitz lemma: if limfc^oo = a and Pk > 0, then 

1- J2k=iPka'k 
lim TT = a. 



□ 



The behavior of Vn /rjN, ss N ^ oo can be just about anything. Take pk = 0, Hjk = 0, 
j > I. Then, 
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— With j3k = l/k, Vk = k'^, and f^ik = k, we get t/tv = and 

Vn aN^ 

— ~ — > oo, Jy oo. 

m In^ N 

for some a > 0; recall that, for a„, 6„ > 0, notation a„ ~ 6„ means 

lim (an/bn) = 1. 

n^oo 

— With = k, = k'^, and /ii^ = A;, we get r/jv = and 

Vn ~ 4r/Ar > ?7Ar 

— With Pk = 1) i^fc = and fiik = {—l)^k, we get rj^ = and 

Vat ~ (In^ 2)7]N < Vn 

— With /3fc = 1, i/fc = k, and = (— we get t/tv = and 

^ n AT 
— ~ — ^ 0) A* ^ oo. 

m N 

Next, we consider Aitken's method. This method consists in transforming a 
sequence A = {an, n > 1} to a sequence 

(On+l — fln)^ 



bn{A) = a 



an+2 — 2an+l + CLn 



The main result concerning this method, is that if hm^ a„ = a and 



lim '""+^ = Ae (0,1), (3.15) 

Or, — a 



then lim„^oo bn{A) = a and 

lim = 0. 

|a„ — a| 

That is, the sequence bn{A) converges to the same limit a but faster. 
Accordingly, under the condition (I3.10p . we define 



with a hope that 



9, = e,- ^ (3.16) 

0k+2 + 26*^+1 — 9k 



,im < 1. (3.17) 
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In general, there is no guarantee that this will be the case because typically rjk ~ ak'^ for 
some a > and 6 > 0, and so, if we set 



we get by Theorem 13.11 



lim ~ °' = 1. 



n^oo — a\ 

Direct investigation of the sequence 6k is possible if there is only one Wiener process W = 
W{t) driving the equation, that is, fijk = for j > 2, > 1. In this case, (|3.8p shows that 

S,^S„^^L- ). (3.18) 

T \ rk+2 - ^Tk+i +rkj 

where = fJ-ik/'^k- Then direct computations show that 
— if rfc ~ ak~^ , a, 6 > 0, then 



IE(0fe - ^o)' 1 



— if r/c = (-l)'^//c, then 



~ TTT, C > 0. 



For more than one Wiener process, we find 

Bk = eo + ^, 

where (Cfc,Cfc) is a two-dimensional Gaussian vector with known distribution. The analysis 
of this estimator, while possible, is technically much more difficult and will require many 
additional assumptions on fijk- We believe that this analysis falls outside the scope of this 
paper, and we present here only some numerical results. We suppose that Fourier coefficients 
Uk satisfy (|3.3|) with uj^ = k,pk = 0, = {—\)^/{k + j), the noise term is driven by 
n = 10 Wiener processes, and the true value of the parameter 9q = 1. From (j3.7p we note 
that the estimates 9j, can be calculated if we only know the value of \og{uk{T) / Uk{^)) ^ 
rather than the whole path Uk{t), < t < T. Using the closed- form solution of equation 
([331) Ukit) = Ufc(O) ex.p{-{9oUk + 'Ej ^^fc/^)* + fJ-jkWj{t)), we simulate log(nfe(r)/ufe(0)) 
directly, without applying some discretization schemes to the process Uk{t). Three type of 
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estimates are presented in Figure 1. The obtained numerical results are consistent with above 
theoretical results: Aitken's method performs the best, Weighted Averages Estimates 
with = k perform better than simple estimates. 




0_85LLL 1 1 1 1 1 1 1 1 

2 4 6 8 10 12 14 16 18 20 

Fourier Coefficient 



Figure 1. Performances of three type of estimates: Simple, Weighted Averages and Aitken's method 



4. Closed-form Exact Estimators 

In regular models, the estimator is consistent in the large sample or small noise limit; 
neither of these limits can be evaluated exactly from any actual observations. In singular 
models, there often exists an estimator that is consistent in the limit that can potentially 
be evaluated exactly from the available observations. Still, no expression can be evaluated 
on a computer unless the expression involves only finitely many operations of addition, 
subtraction, multiplication, and division. 

Definition 4.1. An estimator is called closed-form exact if it produces the exact value 
of the unknown parameter after a finite number of additions, subtractions, multiplications, 
and divisions performed on the elementary functions of the observations. 
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Closed-form exact estimators exist for the model (|3.ip if we assume that the observations 
are Uk{t), k > I, tG[0,T]. 

As an illustration, consider the simple example 

du - 6uxxdt = {u/2)dt + udW{t), 

where x S (0, vr) and zero boundary conditions are assumed. 



With hk = sin(A;a;), we find 

duk{t) = -k^Quk(t)dt + {uk/2)dt + Ukit)dW{t). 
Set Vk(t) = ln(ufc(t)/tifc(0)). Then 

dvk{t) = -k^edt + dW{t). 

In particular, 

vi(T) = -9T + W{T), V2iT) = -AST + W{t) 

so that 

^_ viiT)-V2{T) 
3T 

or 

^=J-ln^^4^. (4.1) 
3r ui{0)u2{T) ^ ' 

Notice that given ui, . . . ,U]\f, we have N{N — l)/2 exact estimators of this type. 

If there are two Wiener processes driving the equation, then we will need three different 

Uk to construct an estimator of the type (j4.ip . The general result is as follows. 

Theorem 4.2. In addition to conditions of Theorem \3.1\ assume that there exist two finite 
sets of indices {k\^ fcj; ■ ■ ■ > ^n); ^ = 1; 2, and a positive integer p such that 



1=1 e=i 
Then there exists a closed-form exact estimator for 9 . 

Proof. Let Vk{t) = ln{uk{t)/uk{0)). From (|3.3p . by Ito's formula, we get 

dvk = -(pk + Ouk + ^Mfc) dt + Y, HkdWj (t), (4.2) 



i>i 



20 



Ig. Cialenco, S. V. Lototsky 



and then 




l^pk}Vq{T) 
f^pkjiPkj + ^^42)) 




(4.3) 



which completes the proof. 



□ 



If there are no Wiener processes driving the equation, then the extra condition of the 
theorem can always be ensured with n = no + 1, because every collection of n vectors in an 
n — 1-dimensional space is linearly dependent. While relation (j4.3p gives a closed- form exact 
estimator, the resulting formulas can be rather complicated when the number of Wiener 
processes in the equation is large; if this number is infinite, then the estimator might not 
exist at all. For comparison, the complexity of the maximum likelihood estimator (j3.7p does 
not depend on the number of Wiener processes in the equation. As a result, when it comes 
to actual computations, the closed-form exact estimator is not necessarily the best choice. 
On the other hand, the very existence of such an estimator is rather remarkable. 

We conclude this section with three examples of closed- form exact estimators. The first 
example shows that such estimators can exist for equations that are not diagonalizable in 
the sense of Definition 12.11 

Example 3. Consider the equation 



where v solves the heat equation vt = Ovxx, v{0,x) = u{0,x). Assume that u{0,x) is a 
smooth compactly supported function. Then u{t, x) is a smooth bounded function for all 
t > 0,x G M and E J^\u{t, x)\Pdx < 00 for all p > 0, t > 0. In particular, the Fourier 
transform U{t,y) of u is defined and satisfies 



du{t, x) = 9uxx{t, x)dt + u{t, x)dWit), <t<T, x G M. 



By the Ito formula. 



u{t, x) = v{t, x) ex.p{W{t) - {t/2)) 



dU{t,y) 



ey^Uit,y)dt + Uit,y)dW{t). 



Let V{t) = \n{U{t)/U{0)). Then 



V{T,y) 



y^QT - {T/2) + W{T) 
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and 

V{T,yi)-V{T,y2) 

nvi - yf) • 

The next example shows that conditions (j2.3p and ()3.10p are not related to the existence 
of a closed- form exact estimator. 

Example 4. Consider the equation 

du - (An + 9u)dt = (/ - Af/^udW{t) 

on (0, vr) with zero boundary conditions. Clearly both (j2.3p and (jS.lOp are not satisfied. 
While the equation is not parabolic, there exists a unique solution in weighted Wiener 
chaos spaces, and we can therefore consider 

duk = {-k^Uk + euk)dt - (1 + k^f'^UkdW{t). 

For Vk{t) = \n{uk{t) / Uk{^)) we find 

VkiT) = i-k^ - ^ \ ' )T + eT+{l + k^ f/^Wit). 

In particular, 

vx{T) = aiT + hxWi^t) + 9T, V2{T) = agT + b2W{T) + OT, 

and so 

_ biV2(T) - b2Vi{T) - (0261 - 01^2)^ 
T{hi - 62) 

The last example shows that, as long as there is no spacial structure in the noise, 
multiplicativity of the noise is not necessary to have a closed- form exact estimator. 

Example 5. Consider the equation 

du{t,x) = 9uxx{t,x)dt + dW{t),t > 0, x G (0,7r), 

^2/7r cos((A; — l)x), 



with Neumann boundary conditions, so that hi = l/\/vr and hk = 
k >2. Then du2{t) = —9u2{t)dt, and, as long as U2{0) / 0, we have 

^=lln."^(«) 



T U2{Ty 
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